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ABSTRACT 

These four papers focus on a consideration of the 
costs of evaluation, as examined in the Evaluation Use Project. "A 
Framework for Estimating Evaluation Costs," by Marvin Alkin and Brian 
Stecher, considers overall theory and draws on previous research in 
determing evaluation costs. The concepts are extended to develop 
informal quidelines for a two-stage cost estimate process. The 
succeeding papers present actual case studies of the cost estimate 
process in three different evaluation contexts within local school 
districts. "The Cost of a District-Wide Testing Program," by N. James 
Myerberg, identifies the specific applicable cost parameters and 
suggests a method to compare testing across school districts. "The 
Incremental Cost of Conducting a Summative Evaluation of a Special 
Education Program From an Existing Data Base," by William T. Denton, 
examines a computerized system. "The Cost of a School Level Needs 
Assessment," by Floraline Stevens, discusses cost estimates according 
to school size and staff familiarity with the process. The question 
of assigning value to the extensive time expenditures required in the 
procedure is examined. Each paper proposes a high and low cost 
alternative to the evaluation procedure and examines the differences 
between options. (CM) 
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INTRODUCTION 



This series of papers represents the November 1982 deliverable 
for the Evaluation Use Project of the Center for the Study of 
Evaluation. These papers focus on a consideration of the costs of 
evaluation. 

The papers are organized first by an overall theoretical frame- 
work for considering evaluation costs. This paper, written by Marvin 
Alkin and Brian Stecher, draws on previous research in the area of 
determining evaluation costs and then extends these cost notions to 
develop informal guidelines for costing out a particular evaluation 
study. 

The three succeeding papers go on to present actual case studies 
of the application of these costing guidelines to three different 
evaluation contexts within local school districts. The first paper, 
by N. James Myerberg, presents a costing exercise relative to a 
large-scale, district-wide testing program. Myerberg identifies the 
specific cost parameters applicable to a testing program and goes 
on to suggest a method for using this costing model to compare testing 
costs across school districts. 

The second paper, by William T. Denton, discusses the costs 
associated with the use of a computerized information and reporting 
system for a district-wide special education program. The costs of 
using a computerized system as compared to a traditional manual 
data tracking system are outlined. Finally, Denton reviews the 
overall cost-effectiveness of computerized data systems. 

The final paper, by Floraline Stevens, looks at the costs of 
performing a school-level needs assessment in a large size metropolitan 
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school district having widely varying school enrollments. Each step 
of the needs assessment process is costed out according to the 
size of the school site involved and the familiarity of school staff 
with the needs assessment process. The paper also examines the 
question of assigning value to the extensive time expenditures 
required for this type of evaluation procedure. Stevens concludes 
with guidelines for self-determination of evaluation costs through 
decisions made about the depth of information required, the nature 
and extent of the personnel involved and the use of data processing. 

Taken as a whole, these four papers provide a sound theoretical 
framework for estimating evaluation costs and provide some carefully 
developed examples that will serve as baseline data for the growing 
discussion of the costs of education. 
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A FRAMEWORK FOR ESTIMATING EVALUATION COSTS 

Marvin C. Alkin 
Brian Stecher 

Center for the Study of Evaluation 

UCLA 



Introduction 

In an era of declining educational budgets, school dis- 
tricts have been forced to examine expenditures with an ever 
more critical eye. Not only are "frills" feeling the sharp- 
ness of the budget-cutting axe, but many previously "essential" 
services are being reduced as well. Given this prevailing 
climate, evaluators would be short sighted if they failed to 
examine thoroughly the cost of evaluation. 

Most people have a "common sense" notion that equates 
program costs with dollars appearing on a ledger sheet at 
the end of the life of a program, accounting costs. But 
economists remind us that such a conception of costs is quite 
narrow. As Levin (1975) explains, "The explicit expenditures 
associated with any particular course of action represent 
only a partial measure of the total costs." For example, 



A paper presented at the Annual Meeting of the American 
Educational Research Association, New York, March 20, 1982. 
The research which is the subject of this report was sup- 
ported in whole or in part by the National Institute of 
Education, Department of Health, Education, and Welfare. 
However, the opinions expressed herein do not necessarily 
reflect the position or policy of the National Institute of 
Education, and no official endorsement by the National Institute 
of Education should be inferred. 
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client time has a value that ought to be considered in ap- 
praising the total social costs of an activity. Similarly, 
the time teachers contribute while administering a test and 
the time students spend taking a tC3t are parts of the total 
cost of the evaluation. Levin (1975) stated it this way— "In 
this sense the term 'costs' refers to the monetary value of 
all the resources associated with any particular action, and 
their value is determined by their worth in the most pro- 
ductive alternative applications." In Haller's (1974) words, 
"This notion of costs as benefits foregone — opportunity 
costs — provides an alternative starting point for anyone con- * 
cerned with costs in educational program evaluation." 

In some instances, however, dollars may be an appropri- 
ate measure of the opportunity costs of a given resource. 
To paraphrase Haller, when we use money as a measure of 
evaluation costs, we are assuming that the dollar figure will 
serve as an adequate measure or description of the resources 
required and the value of the programs foregone. While we 
may choose to make this simplification, we should recognize 
that it is a simplification. As Haggart noted (1978), "We 
are using costs as a somewhat abstract, but common or uni- 
versal measure of the nature and quantities of the resources 
required for a project. We are mostly concerned with the 
costs as measured by using standard prices for the same re- 
sources." This simplifying assumption seems appropriate 
since this symposium has a different purpose — to examine the 
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range of costs of specific evaluation activities as they 
might be conducted by a typical school district evaluation 
office. From this exercise we hope to derive some baseline 
figures that have comparative value for other school evalua- 
tion professionals . 

Literature on the Cost of Evaluation 

Few, if any, published articles present cost data for 
evaluation activities, even though costs of evaluation are 
prominently mentioned in the literature. Scriven's reference 
(1974) to "cost free" evaluation, for example, implies that 
evaluations should more than pay for themselves in the benefit 
they provide to programs, yet there are no suggestions as to 
how these evaluation costs should be calculated* 

The Standards for Evaluations of Educational Programs, 
Projects, and Materials, developed by the Joint Committee 
on Standards for Educational Evaluation include a statement 
relating to the costs of evaluation: "The evaluation should 
produce information of sufficient value to justify the re- 
sources expended." The guidelines clearly indicate that one 
should thoroughly investigate costs and benefits of evalua- 
tions before deciding to do them and that the evaluator "con- 
duct evaluations as economically as possible." However, 
they offer no definition of what amounts or percentages 
should be spent for evaluation or what basis should be used 
for making cost allocations within the evaluation budget. 



There are some informal guidelines for how much evalua- 
tion should cost in terms of the overall program budget — and 
they vary widely. Rusnell (1979) says, "Ten percent of pro- 
gram cost is sometimes suggested for use in evaluation." 
One source of this "common sense" guideline is undoubtedly 
the prescription in the early 1970s that 10 percent of LEA 
Title VII project funds be allocated for evaluation and pro- 
gram audit activities. (To be precise, eight percent was 
set aside for evaluation and two percent for program audits.; 
Another commonly suggested practical criterion is to allocate 
three percent of a program's budget for evaluation. Abramson 
and Wholey (1981) report that HEW evaluations were largely 
funded from one percent set aside funds earmarked for evalua- 
tion in the authorizing legislation of national programs. 

Goldman (1980) presented a functional analysis of the 
New York City Board of Education budget for the fiscal year 
1979-1980. Planning, Testing, Research and Development 
Services received only .7 percent of the total budget of 
the school district during that year. Breaking the services 
down further, Educational Evaluation received .2 percent of 
the funds. In a 1977-78 study of large school districts, 
Webster and Stuff lebeam (1978) noted an average expenditure 
of $6 per pupil for evaluation and associated activities 
(testing and research) . This would appear to be fairly 
similar to the Goldman findings as a percentage of total LEA 
budget. Clearly the percentage allocated to evaluation would 
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be greatly dependent upon such factors as size, cost, and 
type of program. However, these relationships have not been 
studied . 

Drezek et al • (1982), in a survey of 55 LEAs and 14 SEAs, 
examined the evaluation costs as a percentage of total pro- 
gram costs for program proposals. They found a median low 
of 1.5% to a median high of 5.5% across the districts studied. 
Furthermore, their respondents recommended "a 4-8%, sliding 
scale, program cost-allocation for evaluation . " 

Be it one percent, three percent, or 10 percent of new 
outlays for evaluation, such rules of thumb for total evalua- 
tion costs offer little useful information toward developing 
a framework to analyze the costs of various program activities. 
A search of the relevant literature uncovers several studies 
potentially relevant tc the topic of evaluation program costs 
by category. 

Lai (1978) examined empirical cost data on 14 completed 
evaluations whose budgets ranged from $400 to $3 million. 
He hoped to obtain "ballpark estimates of costs expected in 
relation to sample size, number of schools involved, number 
of instruments and items developed and used, report length, 
and project staff time." Unfortunately the evaluations he 
examined varied so greatly on these dimensions as well as in 
subject matter, duration of the evaluation, and the type of 
organization conducting the evaluation, that little useful 
information could be derived. And as Lai concluded, "It was 
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not possible to come up with easily used categories that re- 
lated evaluation parameters to cost." The data might be of 
use to evaluators as representative examples if more informa- 
tion were offered on the scope of each evaluation. 

Paisley et al. (1978) examined the costs of educational 
extension services. They developed a model of normative 
cost estimates tor variously configurated extension services. 
Evaluation was one of eight functional areas of activity 
included. The total program budget projected in their model 
was $250,000 and 6.2 percent was budgeted for evaluation in 
six cost categories: labor — $9,800 (62.9%); material — $1,000 
(6.4%); services (contracted) — $1,200 (7.7%); travel — $600 
(3.8%); occupancy — $1,000 (6.4%); general administration — 
$2,000 (12*8%); for a total evaluation cost of $15,600. One 
should not generalize too far from these estimates, because 
educational extension services have certain characteristics 
that are very different from other educational activities. 
Paisley and his colleagues anticipate wide variations in cost 
estimates within educational extension systems attributable 
to changes in the type of activity; the same comment is valid 
for variations that will occur between extension services 
and other educational organizations. 

Estimating Evaluation Costs 

The review of literature reinforced our initial impression 
that little had been done to examine the range of costs for 
different types of evaluation activities. Similarly , there 
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was little in the literature that could be used as a direct 
guideline for developing an evaluation cost estimation 
framework — particularly as it relates to evaluation in 
school district settings . 

Assumptions , There are, however, sources of guidance 
for costing evaluations. The principles that have been ex- 
pounded for assessing the costs of other programs can be 
used to examine evaluation activities themselves. Most of 
the books on program budgeting in schools have chapters in 
them about costs. Haggart (1979), for example, has provided 
an excellent description of an approach for educational pro- 
gram cost analysis. This book provides basic ground rules 
for costing; one only needs to substitute "evaluation" for 
"project. " 

There are ways, of course, that evaluations are dif- 
ferent from projects in their costing implications. ,The 
elements to which costs are attached are different for evalua- 
tions than for programs generally. In an evaluation, we have 
an entity which, depending upon the way in which it was 
funded, may or may not be partially internal to the school 
organization and partially external. If the evaluation is 
performed solely by an internal district evaluator, then 
cost allocation procedures are straightforward. But, if the 
evaluation is performed by an external evaluator, then there 
are both external costs and associated internal costs. The 
extent to which this is complicated depends upon the procedures 
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employed to deal with external costs and, more specifically, 
on the level of detail desired. We will assume that external 
evaluator costs are under the category of contracted services 
and will not attempt to break them into other categories. 
While an understanding of the allocation of externally con- 
tracted evaluation services is important for other purposes, 
in this paper we will accept this simplifying assumption. 

The very specific context for our work in this symposium 
allows us to make some further simplifying assumptions about 
costs. We can also assume that the district has an existing 
administrative structure and evaluation capability. We do 
not have to start from scratch and include the cost of es- 
tablishing an evaluation unit, installing telephones, or 
erecting classrooms; set up costs and other fixed costs have 
already been covered. Cost estimates may be based on the 
assumption that existing evaluation offices are operating at 
their usual level and that one additional project is being 
undertaken. Thus, we suggest considering only marginal costs. 

A consequence of this assumption is that only the addi- 
tional costs related specifically to the new evaluation 
activity will be included in estimates. That is, there should 
be no indirect charges in the cost estimates. As Haggart 
(in press) directed: 

In estimating the cost to be used in (conducting the 
evaluation) , all resources needed for the implementa- 
tion and operation of the evaluation are included... 
The cost of resources that do not change because 
of the presence of the evaluation are not relevant. 

1 < 
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Thus, for example, there will be no costs for acquisition of 
equipment unless one would be buying new equipment specific- 
ally for this evaluation. Similarly, there will be no costs 
for facilities that already exist, for building maintenance 
that is already taking place, or for other overhead charges 
unless the evaluation directly increases these activities. 

As the previous quote from Haggart suggests, the re- 
source requirements determine the cost of the evaluation. 
In fact, resource needs may be a more fundamental measure of 
the administrative impact of the evaluation than costs. 
(Costs only measure the local value of the resources, and 
this will vary from district to district.) Similarly, the 
resource requirements themselves derive directly from the 
scope or description of the evaluation — what is being done. 
This analysis suggests that. 

After describing the scope and general nature of the 
evaluation, there are two steps which are fundamental to 
estimating the cost of evaluation: 

1. Delineating the resources required to carry out 
these processes ; and , 

2. Assigning dollar values to these resources. 
Resources must be specified precisely as a part of any 

costing exercise for several reasons. Haggart (1978) notes 
that there are differences between the cost of resources in 
different communities. One can either select to use local 
dollar costs or one can use resources as a standardized 
measure that approximates costs more generally. The choice 
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depends on the purpose for which one is making the cost 
estimate. Standard costs are appropriate if one is going to 
use the cost estimates to compare evaluation in different 
districts; local costs are critical if one is trying to 
determine whether or not a particular activity should be 
conducted in a particular site. It was easier for the par- 
ticipants in this symposium to use local costs. However, to 
insure that the results have wide applicability, each of the 
papers in this symposium will present both resource statements 
and present dollar values. 

Cost Categories 

Another element that must be standardized is the cate- 
gories used for aggregating and reporting cost figures. Levin 
(1975) suggests five major categories of resources: per- 
sonnel, facilities, material and equipment, other, and the 
value of client time and other client inputs. Each of these 
categories can be broken down into a number of subcategories. 
For example, personnel could be broken down into certificated 
personnel and classified personnel. Further subgroupings 
may yield greater detail but not necessarily greater accuracy. 

The aategories should reflect the inherent nature of 
the activity that is being analyzed. Sanders (1982) used 10 
categories when he assessed the cost implications of the 
standards for program evaluation. Haggart et al. (1978) 
used only three categories. Our experience with educational 
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evaluation suggests that resources could be best grouped in 
the following seven categories : 

1. Professional personnel (salaries and benefits 
together) 

2. Clerical and secretarial personnel (salaries and 
benefits together) 

3. External consultants 

4. Materials , equipment , and supplies 

5. Data processing 

6. .Facilities 
1. Other 

At this point in our understanding of evaluation costs 
our choice of categories must be somewhat subjective. For 
example, we separated data processing costs from materials 
and supplies because they are an essential part of many 
evaluations, and they are a service that often exists as a 
functionally distinct activity . 

We have not included the value of client time and other 
client inputs on our resource list for several reasons* 
First, it is commonly assumed (though this may not be war- 
ranted) , that the small disruption in the instructional pro- 
gram caused by testing and other evaluative activities has 
only minimal impact. Moreover, we are focusing primarily 
on direct dollar costs of evaluation and thus, as we have 
noted, this simplifying assumption is warranted. 

Conclusions 

Our purpose in this paper was to develop a framework 
that could be used for making estimates of the costs of 
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various evaluation services. We began with a general dis- 
cussion of the nature of costs, and then have made some 
simplifying assumptions that seemed appropriate to the tasks 
of assessing the cost of educational evaluation in school 
districts. We offered a two-stage process for making evalua- 
tion cost estimates. One first specifies all the resources 
that will be involved in the evaluation, and then assigns 
values in terms of dollars to the various resources. 

This brief costing paper was developed as a framework 
for guiding the three remaining papers in this symposium. 
Each of the three other panelists was asked to describe and 
then "cost out" a particular evaluation activity in their 
school district. As a final activity, we asked each to dis- 
cuss the variability of their estimates. We hoped to do 
this by having each panelist propose a high cost alternative 
(both brief description and total cost) and a low cost alterna- 
tive to the evaluation procedure described in the paper and 
analyze the key differences between these options. 

We hope that this formulation proves to be useful as a 
baseline for an analysis of evaluation costs generally and 
for the papers of this symposium. 
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Th e Cost of a District-Wide Testing Program 
N. James Myerberg 

I . Introduction 

Most school districts today have some type of district-wide 
testing program. Such testing programs have to a great extent 
been influenced by the accountability movement in education. 
A school districts commitment to a testing program is generally re- 
flected by the amount of /money put into the program. This paper 
will discuss the elements of a model to identify a particular dis- 
trict^ commitment to its testing program and will provide a method 
for using this model to compare costs/commitment across school dis- 
tricts . 

Scope and Sequence of a Systemwide Testing Program 

The Montgomery County (MD) Public Schools conduct annually an 
extensive district-wide pupil testing program. This effort requires 
a significant commitment in terms of personnel and support costs 
which need to be systematically included in the district's overall 
budget. A method has been developed / therefore, to frame the cost 
parameters of the testing program in terms of district size, scope and 
sequence of testing, personnel required, and support services. 

The systemwide testing program (95,500 students) consists of the 
administration of two tests: a nationally normed standardized test 
and a state developed minimum competency reading test. The 
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standardized test, the California Achievement Test (CAT), is ad- 
ministered each fall to all students in Grades 3, 5, 8, and 11, 
The minimum competency test, the Maryland Functional Reading Test 
(MFRT) , is administered in October to all students in Grades 7 and 
9. The MFRT is also administered to students in Grades 10, 11, and 
12 who have not previously passed the test, and is administered 
again in May to students who still have not passed. Approximately 
46,900 students were tested in this program during the 1981-82 school 
year. The breakdown of number tested by grade and test is shown 
below. 







Number 


Test 


Grade (s) 


Tested 


CAT 


3 


5,200 


CAT 


5 


6,600 


CAT 


8 


7,300 


CAT 


11 


8,100 


MFRT 


7 


7,600 


MFRT 


9 


7,700 


MFRT 


10-12 


1,900 


MFRT 


9-12 


2,500 



The testing office is responsible for implementing this program, 
Its functions can be grouped into four broad categories : planning 
and preparation for the program, orientation of administrative and 
teaching staffs, processing results, and reporting and explaining 
results. The resources needed to accomplish these tasks and the 
costs attached to these resources are described in the next section. 
It should be noted that the figures presented in this section can be 
considered as representative of the extent of the district's commit- 
ment to the testing program. 
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II. A Model for Deriving Testing Program Costs 



Costa of the Systemwide Testing Program 

The total direct cost of the systemwide testing program for 
the Montgomery County (MD) Public Schools for the 1981-82 school year 
was $162,500- This cost is broken down into several budget cate- 
gories listed below. The listing is followed by a description of the 
resources (and their attendant cost implications) that fit into each 
category . 

A. Budget 
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Cost Category Dollar Amount 

Professional personnel 

(including benefits) 115,500 
Clerical and secretarial personnel 

(including benefits) 23,000 

External consultants 4,000 

Materials, equipment, and supplies 15,000 

Data processing 5,000 

B. Description of Resources 

Professional Personnel . The Coordinator of Testing supervises 
all aspects of the program. He has three assistants whose chief re- 
sponsibilities involve preparation of orientation materials, pro- 
cessing results, and helping to prepare reports of the results. 
During the fall and winter all of these people work full-time on 
testing. In spring and summer, the testing activity lessens and 
they become involved in other activities of the Division of Instruc- 
tional Evaluation and Testing. 

In general, the more professional staff are hired as on-site 
district personnel, the more services are provided to district staff 
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and students, and the better the training provided to district staff 
in the administration of tests and the use of test results. A 
smaller district commitment in terms of professional personnel 
usually leads to the hiring of external consultants, which in turn is 
reflected in a different budget category. If external consultants are 
not hired, district staff will generally lack the required training/ 
expertise/assistance in the administration of tests and in interpret- 
ing the results. 

Clerical and secretarial personnel . More than half of the 
cost in this category results from the necessity for quality control 
of answer sheets. This process involves a review of every answer 
sheet before scanning, and a subsequent correction if it appears that 
sheets may be incorrectly scanned because of such things as light 
marks, poor erasures, or incorrect student identification information. 
Other costs in this category arise from the need for secretarial 
help for production of orientation materials and reports; data 
tabulation, often in response to unexpected requests from the Board 
of Education or Superintendent; and packing, distributing, and pick- 
up of test books. 

If quality control is not provided for in this budget category, 
approximately five percent of student scores may be adversely af- 
fected. Inaccurate scores could have serious educational consequences 
"for the students who are thus misidentif ied . 

External consultants . This cost comes from the use of consum- 
able test books in Grade 3, which have to be scanned by an outside 
scoring service. A district which has an adequate professional staff 
will not need to expend much in this cost category. 

er|c 24 



Materials, equipment , and supplies . The major cost in this 
category is fox* CAT test books. Consumable books have to be purchased 
each year. Non-consumable books represent a one-time purchase but 
the cost has been divided across the expected number of years the 
books will be used. In this case the expectation is six years. 
Another one-time cost is for support materials for the CAT. These 
materials include manuals describing the test and extra examiner's 
manuals needed because the one per 35 test books provided by the 
publisher is not sufficient. The cost of the support materials has 
also been divided across six years. There is no cost for the MFRT 
because it is produced and distributed by the state education depart- 
ment . 

The other items in this category are individual student report 
forms for each test, and adhesive labels and letters for use in re- 
porting results to parents. 

Data processing . This cost is for a programmer who serves as a 
trouble shooter if there is a problem in processing of tests. This 
category does not include costs for things such as computer operators 
and computer paper since they do not represent additional costs for 
the data processing department. It also does not include the cost of 
originally developing the test scoring system, since inclusion of this 
cost would greatly increase the total cost of the testing program and 
provide a misleading estimate of the annual cost of the program. 

A "debugged" system of data processing which is under local 
control provides a district with greatly increased flexibility, since 
report formats can be altered or designed to provide information of 
specific district interest/importance. 
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IIT. A Method for Comparing Testing Costs Across School Systems 

Any comparison of costs across school systems is , obviously, 
confounded by differences in the cost of living in the areas served 
by these districts. While the material costs cited above will remain 
constant across the country, the personnel-related costs will vary 
across districts. A method for standardizing the cost of a testing 
program, or other evaluation activity, would be to relate the program 
cost to each local system budget. The budget will, to some extent,, 
reflect the local cost of living and, therefore, tend to cancel out^ 
the effect of different costs of living. Since a testing program ca^i 
be related to a specific number of students, the cost per student can 
be determined. This cost can then be compared to the total cost per 
student in the system. The percent of this total represented by 
the testing cost can serve as the comparative index. In the case of 
Montgomery County, with about 95,500 students, the per student testing 
cost is $1.70 compared to an overall student cost of $2,982. The 
testing cost is thus determined to be about one-twentieth of one per- 
cent percent of the total. 



-23- 



THE INCREMENTAL COST OF CONDUCTING A SUMMATIVE EVALUATION OF A 
SPECIAL EDUCATION PROGRAM FROM AN EXISTING DATA BASE 

William T. Denton 

Introduction 

The Dallas Independent School District (DISD) conducts a large 
Special Education Program with ancillary services (9244 students 
annually). The DISD is responsible to both Federal and State 
agencies for evaluation of services provided under applicable 
laws (especially PL 94-142). To facilitate the gathering and 
reporting of information, as well as to provide program. evalua tors 
with easy access to student/program data, an extensive computerized 
data base has been developed and implemented across the entire sys- 
tem. This paper reviews the capabilities and functioning of this 
computer data base, and discusses the cost-effectiveness of using 
the computerized system as compared to a manual system for provid- 
ing the same information. 

1. Description of the Computerized Data System 

A. Objectives . The Computerized Information System (CIS) 
consisted of the following three systems: (a) The Special Educa- 
tion Data Base, containing demographic data 'files on Dallas 
Independent School District (DISD) special education students 
and faculty, (b) The Computerized Monitoring and Retrieval (CMR) 
System, used as a means to access and store appraisal/placement data 
when students enter the special education process, and (c) The 
Computerized On-Line Transportation (COLT) System, allowing storage 
and retrieval of information regarding special education 
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transportation services. The basic function of the 1980-81 Special 
Education CIS was to build onto the existing computer systems 
while continuing to provide information to special education ad- 
ministrators. Variations of student and faculty rosters, current 
to the week of distribution, were given to special education coor- 
dinators, facilitators, diagnosticians, home/school coordinators, 
psychologists, ar.d speech pathologists on a six-week basis. 

A major objective of the CIS was the generation of reports, 
necessary for funding purposes, for the Texas Education Agency (TEA) • 
The PL 94-142 Child Count reported the total number of children 
between ages 3-5 and ages 6-21 by handicapping con m. The 
Elementary and Secondary School Civil Rights Survey tallied the 
total number of special education students by age, instructional 
arrangement, and handicap. Finally, statistical tabulations es- 
sential for the Annual Special Education Statistical Report to the 
TEA were generated by a set of computer routines developed by CIS 
personnel . 

The CIS was utilized also to provide information requested by 
management. The types of information requested usually dealt with 
totals and percentages of special education students in terms of 
ethnicity, subdistrict, age, or handicap. Mailing labels were 
also computer-generated, which economized on secretarial time when 
letters were sent to staff or parents. 

The CIS also provided on-line computer storage and retrieval 
of appraisal/placement events. Computer terminals, placed in sub- 
district special education offices, enabled immediate access to 
student information such as dates of reviews, staf f ings , and 
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medical exams. Finally, on-line capability to add or change 
information regarding special education transportation services 
was provided. 

B. Data Collection and Storage . The information for the 
data base was gathered by speech pathologists (N=83) and case 
managers (N=73) . The information collected by case managers, 
also known as diagnosticians, psychologists, and home/school 
coordinators, primarily consisted of student demographic data. 
The information was recorded on forms and sent to a central office 
on a weekly basis. When a student was added to a special education 
program, the student's name, identification number, school, class- 
room unit, handicap, and instructional arrangement were recorded. 
All variables, excluding the name, were written in an alpha or 
numeric code. The same method was used when a student was dropped 
or a change was made on the student's record. After the forms 
were validated, the data were keypunched and entered into the 
computer file. 

The CMR data were entered through remote terminals by area 
administrative clerks (N=14). First, the data were sent to the area 
office on appraisal forms completed by the case managers. The clerks 
input the data on the terminal after recording the information on 
Case Management Profile Forms. 

Information for the COLT System was provided by coordinators 
and facilitators who sent transportation updates to the central 
administration building on a daily basis. Updates were entered 
directly through a remote terminal stationed in the special educa- 
tion office. 
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Figure 1: Special Education Computerized Information System 
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Special Education Data Base 

Student File 

1. Student DISD identification number. 

2. Student name. 

3. Attending school location. 

4. Teacher unit number of speech pathologist number. 

5. Disposition code. 

6. Instructional arrangement code. 

7. Primary, secondary, and/or tertiary handicap code. 

8. Student services number. 

* 9. Home-school location code. 

10. Speech problem type. 

11. Speech handicap severity rating. 

12. Number of speech sessions served. 
13 • Case manager number. 

14. Special flags for students affiliated with Project Zero 
Reject, Project KIDS, Project SPICY, Project SEED, the 
Multiple Careers Magnet Center, or the Assessment Center. 

15. Special flags indicating students who are expectant mothers, 
receive special funding, or are severely profoundly 
handicapped. 

16. Special flags for students in vocational educational 
programs . 

17. Related services (i.e., occupational therapy or physical 
therapy) . 

*18. Home address. 

*19. Phone number. 

*20. Grade. 

*21. Date of birth. 

*22. Sex. 

*23. Ethnicity. 

Faculty File 

1. Area location code. 

2. Position type code (i.e., psychologist, teacher, etc.). 

3. Faculty identification number. 

4 . School location code. 

5. Supervisor identification number. 

6. Diagnostician identification number. 

7. School grade span. 

8. Organizational arrangement code (i.e., DISD or non-DISD 
campus) . 

9. Service type code (i.e., resource room, self-contained 
unit, etc . ) . 

10. Itinerant location codes. 



*Data initially retrieved from the System-wide file. 
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School File 

1. School TEA budget number. 

2. School name. 

3. Area location code. 

4. School box number. 

5. School grade span. 

The information on the CMR System contained the following: 
Computerized Monitoring and Retri eval System 

1. Student DISD identification number. 

2. Student name. 

3. Case manager code. 

4. Student services number. 

5. Reporting school. 

6. Referral source. 

7. Dates for PPC (Pupil Personnel Committee) documentation 
and completion. 

8. Date for parent permission of assessment. 

9. Date for scheduled assessment. 

10 . Examination dates (i . e . , vision , auditory , psychological , 
educational physical, and sociological exams). 

11. Dates for scheduled and completion of staffing. 

12. Indication of parent's attendance at staffing. 

13. Results of staffing (e.g.. Was the student eligible for 
special education?) . 

14. Date of parental approval for service. 

15. Location of service initiation. 

16. Dates of area and central ARD (Admission, Review, and 
Dismissal) reviews . 

17. Results of reviews (e.g., Was ARD Committee in agreement?). 

18. Dates of quarterly IEP (Individualized Educational Pro- 
. gram) reviews. 

19. Date and result of end-of-year review. 

Finally, the data collected for the COLT System included the 
following: 

Computerized On-Line Transportation System 



1. 


Student DISD Identification number . 


2. 


Student name. 


*3. 


Pick-up address. 


*4. 


Home phone number. 


*5. 


Sex. 


*6. 


Ethnicity. 


*7. 


Date of birth. 


8. 


Student services number. 


9. 


Transportation location. 


10. 


Transportation funding source. 



Retrieved from System-wide Data Base. 
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Computerized On-Line Transportation System - Cont'd. 

11. Transportation status code (i.e., parent request, transporta 
tion started, or transportation stopped) . 

12. Bus route number. 

13. Transportation eligibility reason (i.e., use of wheelchair, 
braces , etc . ) . 

14. Indication of the necessity of a special lift. 

15. Delivery address. 

16. Work phone number. 

C. Reports Generated . Table 1 categorizes the frequency . and 
types of reports generated on a routine basis. Faculty, student, and 
transportation rosters were generated every six weeks. Summary 
statistics on the CMR System were generated every month. Informa- 
tion was reported annually to TEA, and data necessary for the 
scheduling process were printed also on an annual basis. 

Information from the CIS was requested on an ad hoc basis on 
162 different occasions during 1980-81. The number of ad hoc re- 
quests increased by 14 percent when compared with the 1979-80 re- 
quests (N=142). The time needed to complete a request ranged from 
15 minutes to four days. The average response time for a request 
was 2.10 hours. 

Professionals needing information from the CIS ranged from 
assistant superintendents to subdistrict administrative clerks. 
The number of ad hoc requests from central staff (N=66) increased 
by 20 percent when compared with 1979-80 central staff requests 
(N=55) . This group accounted for almost half of the ad hoc requests 
during 1980-81. There was a 30 percent decrease in the number of 
requests by the Research and Evaluation staff from 1979-80 (N=42) 
to 1980-81 (N=32). Another comparison of 1980-81 to 1979-80 re- 
vealed that a greater range of people requested data. The 198 0-81 
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Table 1 



Types and Frequencies of Routine Data 
Reported from the Computerized Information System 



Type of Report Schedule for Generation 

CMR Summary Statistics Monthly 

Bctculty rosters 6 weeks 

Student rosters « 6 weeks 

Transportation rosters 6 weeks 

Child Find Report Quarterly 

Elementary and Secondary School 

Civil Rights Survey Annually (October) 

P. L. 94-142 Child Oount - Annually (December) 

Annual Special Education Statistical 

Report to the TEA Annually (June) 

Letters of notification to parents 

about special class placenent and 
transportation authorization, 

if applicable Annually (June) 

Conputer punch cards of students 

necessary for class scheduling 

process Annually (June) 
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school year marked the first year of requests by home/school co- 
ordinators, psychologists, and school personnel such as registrars 
and counselors . 

D. Categories of Information Use , Utilization of CIS informa- 
tion included several major areas: class scheduling, enrollment 
projections, ad hoc information requests, and on-going information 
retrieval. The following summarizes CIS utilization in each of these 
areas during 1980-81. 

A computerized method of class scheduling for the 1981-82 
school year was implemented in the summer months. The method, which 
minimized the work time of special education management considerably, 
was executed with the close cooperation of the coordinators. 

A set of programs was updated from 1979-80 to project next- 
year school locations for special education students based upon their 
neighborhood elementary schools and the District-wide school feeder 
system. The enrollment projections were designed to help management 
allocate class units relative to anticipated student enrollments in 
the annual budgeting process of the District. 

Subdistrict personnel were able to access current information 
on computer terminals placed in the area offices. Two programs, 
Tabulator and Inquiry, were specifically designed for users with 
minimal data processing background. These programs provided ac- 
curate and current statistical and demographic information on the 
special education population. 

The additional functions of and revisions to the 1980-81 CIS 

were : 
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1. Collection of related services and vocational information 
by case managers on an on-going basis. 

2. Collection of data on expectant mothers on an on-going basis. 

3. Generation of new software necessary for: 

a. The Computerized On-Line Transportation System. 

b. The Computerized Monitoring and Retrieval System. 

c. The student directory roster. 

d. The Extractor and Tabulator programs. 

4. Addition of the multiple updating feature on the CMR 
System which allowed all subdistricts simultaneous up- 
dating of the System. 

II, Determining the Costs of the CIS 

A. System Costs . The cost of CIS operations was ascertained 
by dividing the system into six major functions and charting the 
hour costs utilized to perform each. Table 2 indicates the total 
number of days spent on each CIS function given two computer pro- 
grammers, one assistant evaluator, one principal evaluator, one 
secretary, and one data technician. Assignment of cost to each 
function was achieved by totalling each employee's daily salary 
multiplied by the number of days utilized for each function. 

Keypunch and computer time costs were not included in Table 2, 
but the cost incurred for keypunching totaled $1,502.48. A previ- 
ously reported analysis of the costs of data processing revealed 
that, at least in a large school district which has computer capa- 
bility, the costs for computer-prepared individual reports were 
less than those costs for reports which were manually prepared 
(Reisman, Holt, Kocsis, and Macy, 1979, p. 6) . 

When compared with the 1979-80 cost analysis, more time and 
money were devoted to software development, data collection, and 
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Table 2 

Computerized Information System 
Cost Analysis a 





Total Days 




Percent Total 


CIS Function 


Utilized bv Staff 




fV\ct- 
\JOSf. 


New software development** 


257 


428,893.20 


40.4% 


Data collection 


162 


$13,781.50 


19.3% 


System maintenance 


66 


4 9,143.10 


12.8% 


Special projects^ 


80 


4 8,558.70 


12.0% 


Routine report generation 


59 


4 6,084.60 


8.5% 


Ad hoc report generation 


46 


4 4,983.90 


7.0% 


Total 


670 


471,445.00 


100.0% 



a Cost figures do not include computer" time. 

**few software was developed for the CGLT System, CMR System, and for data 
base improvements. 

c Special projects included the class scheduling process, the development 
of an enrollment projection system, and the study of standardized test 
scores of special students. 



37 



-34- 



system maintenance in 1980-81, while less time was utilized for 
routine report generation and special projects. Specifically, 
resources allocated for software development doubled in 1980-81 
due to revisions to the CMR System and implementation of the COLT 
System. Adding the multiple update feature to the CMR, i.e., 
allowing all subdistricts simultaneous updating to the system, 
required 30 days of programmer time. More resources were alloted 
for data collection and system maintenance because of CIS expan- 
sion. Specifically, the three systems (CMR, COLT, and data base) 
were updated and maintained on a continual basis, whereas, in the 
past, only the data base was in full-time operation. 

B. Evaluation-Related Costs 
Size and scope of the evaluation . The State of Texas requires a 
rather thorough accounting of special education that culminates 
with a final evaluation report. the purpose of the evaluation 
is to account for District efforts toward meeting the mandate 
that every handicapped child shall have a free, appropriate 
education in the least restrictive environment. 

Every year approximately 10,000 students are served in 
special education programs in the Dallas Independent School 
District. Extensive data are collected and stored on each 
student as described previously. 

In addition to the student file, there are also files on 
faculty, schools, and interactive files for monitoring student 
assessment and placement as well as transportation. The annual 
state report is produced by analysis of selected factors within 
these files. 
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Required resources. The personnel involved in generating the 
annual state report include two computer programmers, two evalua- 
tors, and one secretary. Data processing is accomplished with one 
keypunch machine, one remote job entry facility, and seven 
terminals with modems. The main frame computer is a Borroughs 
6700. Since all equipment is shared with other services and would 
be needed if special education did not use them, their cost is 
not included in this cost-analysis. 

C. Cost-Effectiveness of the CIS 
Dollar value of resources . The administrator responsible for 
preparing the annual report made a comparison of cost of preparing 
the report with and without the aid of a computer. This comparison 
is presented in Table 3. 

Using dollars adjusted for inflation, the 1980-81 personnel 
costs for a computer-generated report had actually declined to 
$15,048, reflecting that District salaries had not kept up with 
inflation. Using the categories suggested for this presentation, 





Table 3 






Cost Comparisons 




Manually Prepared 
1973-74 Annual Report 


Computer Generated 
1977-78 Annual Report 


Number of reported 
students 


9,193 students 


9,244 students 


Staff time expenditure 
for report preparation 


4 months 


1-1/2 months 


Salary costs 


$21,651 


$15,514 
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the breakdown for the total cost of the annual report would be: 



One thing I would like to point out in closing is that per- 
sonnel costs for all evaluations are greatly affected by inflation 
and personnel policies. Differences due to inflation can be ac- 
counted for from district to district. Personnel policies such as 
a promotion of a key staff member or salary differentials from 
district to district are much harder to reconcile. Further, at 
least in data-based evaluation, some of the costs attributed to the 
annual report should actually be prorated in some manner across all 
of special education. Certainly, these basic data are used by all. 



Professional Personnel 

Secretarial Personnel 

Materials, Equipment and Supplies 



$13,898 
1,150 
165 



Total 



$15,213 
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THE COST OF A SCHOOL LEVEL NEEDS ASSESSMENT 

Floraline Stevens 
Los Angeles Unified School District 

The Los Angeles Unified School District has 713 school sites 
serving more than 540,000 students in grades kindergarten through 12. 
All of these sites, in some manner, go through the needs assessment 
process to determine what are instructional and educational needs of 
the students. However, the formally mandated needs assessment process 
only occurs in those schools that have special funds from federal or 
state agencies. The Compensatory Instructional Programs Division 
office in the district requires its more than 200 schools to go 
through a needs assessment each year. The state's over 300 School 
Improvement Program schools have the same requirements. The district's 
bilingual program, partially funded by "che State of California, in 
many instances overlaps the compensatory educational programs or 
school improvement programs, and encompasses many more schools that 
require needs assessment information. Since there is not a slot in 
most budgets for a needs assessment , the costs involve a realloca- 
tion and redistribution of staff time and resources. Also, some of 
these costs are imbedded in the myriad of tasks of central administrative' 
staff and evaluation staff as they provide support services that im- 
pact school staffs in the need assessment process. 

Before analyzing the costs of a school needs assessment process, 
a review is in order. The needs assessment process is fundamental 
to program planning for it provides the basis for program develop- 
ment. The effectiveness of the planned program is dependent upon 
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th e thoroughness of the assessment process. A needs assessment 
process involves collecting data and analyzing the data collected. 
(This analysis contrasts the current effects of the program on 
students — e.g., student achievement or student attitudes — with de- 
sired effects, and examining the program to identify what seems to 
be causing the observed effects.) 

The steps in the needs assessment process generally include : 

1. Identifying the goals (student oriented) . 

2. Ranking the goals. 

3. Assessing the performance level of the goals. 

4. Establishing the priority of each goal according to im- 
portance and performance. 

5. Analyzing the reasons for the discrepancies among prior- 
ities, performance, and current program. 

6. Identifying the goals for support components as they relate 
to needs of students. Then repeating steps 2-5. 

When attempting to ferret out the costs of needs assessment 

at the school level, there are several components or functions that 

warrant examination. One way to determine cost is to tally the 

hours expended per task, multiplied by the average salaries of the 

persons involved in the needs assessment activities. A gross cost 

figure then can be given, to the process. The components which I 

feel influence the school level costs are the following: 

1 . Size of school 

2. Depth of information needed 

3. New or continuing process (staff familiarity with process) 

4. Availability of information 
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Size of School 

In the Los Angeles Unified School District there is an ad- 
ministrative policy about off -norm persons allocated to serve a 
school. In an elementary school under 950 pupils, there will be only 
a principal assigned. If enrollment exceeds 94 9 , one assistant 
principal is assigned; over 1349, two assistant principals, and 
over 1799, three assistant principals. Large schools can use the 
assistant principal (s) to coordinate the needs assessment activities 
and thus provide more time for the process. in those elementary 
schools with only one administrator, less time might be allotted 
because the principal's time must be devoted to other administrative 
tasks. In those schools with special funds to pay for nonteaching 
teacher positions, the needs assessment tasks can be given to a 
teacher rather than an administrator. This allows for a more in- 
depth needs assessment. 

Depth of Information Solicited for a Needs Assessment 

Most needs assessment "how-to" documents recommend that ad- 
ministrators and teachers, aides, students, parents, and the community 
be solicited for their input on the needs of a school. Based upon 
the size of the school and its human resources available, some of 
these persons may not be included. However, programs that are 
federally and state funded are mandated to include parents in the 
process. Student data can be limited by using student test scores 
and not the students 1 ranking of needs. Some schools may elect to 
use samples of various populations to rank the needs, while others 
will attempt to garner rankings from the entire population. 
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In most instances, it is the decision of the administrator 
in charge of the needs assessment to determine the depth of informa- 
tion to be gathered. However, in order to make sure that schools 
gather enough information to make program plans, the Los Angeles 
Unified School District's Research and Evaluation Branch has pre- 
pared a Needs Assessment Guide (1981) for schools to use when 
planning their needs assessment. The guide covers the following 
areas: (a) needs assessment process, (b) data from the instructional 
components, (c) assessment of program elements, (d) instructional 
support components/areas, and (e) random sampling. 

Needs Assessment as a New or Continuing Process (Staff Familiarity 
With the Process) ~~ 

Costs are determined a great deal by whether or not the needs 

assessment is a new or continuing process. A continuing process 

takes less time because the information in many instances has to be 

refined or only missing pieces of information supplied. The staff, 

in many instances, does not have to receive inservice training when 

the process is continuing. However, when the process is new to the 

administrator or new to teachers, much time must be spent explaining 

the reason for a needs assessment, the processes involved, and the 

plan to conduct the needs assessment in the school . 

Availability of the Information 

Those schools involved in the ongoing evaluation process would 
have an easier time conducting a needs assessment because many of 
the needs have already been discovered. Those schools not involved 
in such a process would have an extra step to perform — that is, 



15 



determining what are the discrepancies between planned outcomes and 
actual outcomes. Again, the need for the process to occur demands 
additional hours of work by school staff. 

Typical Needs Assessment Cost for a Small School 

In an attempt to determine costs of an actual needs assessment , 
an elementary school principal was interviewed for his perceptions 
of the costs to his school for a needs assessment. The school ±9 
Title I funded and has 700 pupils , 30 teachers, 55 educational 
aides, two teacher coordinators, and one principal. He estimates 
the costs for needs assessment in this manner. 

Teachers . All teachers at a staff meeting review the programs 
and budgets. Then the teachers rank the needs. This takes one 
hour per teacher to perform this portion of the needs assessment. 
Thirty hours » 

Educational aides . All aides go through the same process as 
the teachers . Fifty-five hours . 

Principal and teacher coordinators . The principal attended 
an inservice training session in the summer on needs assessment 
for four hours given by the Research and Evaluation Branch. The 
principal and teacher coordinators prepared the assessment forms 
and data sheets about the school's programs and their budgets. 
In addition, they tallied all forms and recorded the results. 
Thirty hours . 

Parent councils and community persons . Needs assessment forms 
were sent home to all parents and given to the members of the school 
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advisory council. In addition, forms were sent to community persons 
who might have an interest in the school. Over 350 forms were 
returned to the school. Three hundred fifty hours . 

Results . A total of 465 hours were expended for the school's 
needs assessment. Of the 465 hours, 115 were reallocated from time 
that would be used by the school staff for other tasks. If the 
costs were to be generated into dollar values, there were five 
teacher days, nine aide days, and 8-1/2 administrator days. 
Parent/community hours would not be assessed as school costs 
other than the time needed to tally the forms. As previously stated, 
the average per hour paid to each employee classification could 
generate a gross cost figure for the total needs assessment process. 

The costs of a needs assessment are great. If all hours are 
combined, a school month of time is allotted to the needs assessment 
process in a small school by Los Angeles standards. Obviously as 
the number of students and staff increase, the needs assessment 
costs will also increase. 

In sum, the cost of a given needs assessment comes from the 
depth of information to be collected. If a needs assessment form is 
standardized across all schools, the depth of information is low 
but so is the cost. If ranking can be centralized through the use 
of data processing, the costs will also decrease. If, however, a 
school with a large number of teachers and aides is involved, and 
if students are also included in the needs assessment process, 
then the costs of the needs assessment as far as the hours expended 
are concerned increase tremendously. 
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